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media source to a network server module that is communi- 
cating to the plurality of client terminals. The proxy module 
is comunicatable with the network server module to be 
signaled a switch request. In response to the switch request, 
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the network server module is signaling streaming media 
from the first streaming media source to the plurality of 
client terminals. 
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VIDEO AND AUDIO STREAMING FOR The real audio system requires that not only the client 

MULTIPLE USERS have additional software, but that the content provider have 

a separate server from their normal web server. For a client 

CROSS REFERENCE TO RELATED rwcive a real audio broadcast, the client typically con- 

APPLJCAnONS 5 jjecis through their browser to a Web page with a reference 

This continuation-in-part appliction claims priority from to a real audio server. The client then accesses its separate 

the U.S. patent application Ser. No. 09/001,850, filed Dec. real audio player program. The real audio player program 

31, 1997, entitled \^eo And Audio Streaming For Multiple then connects to the referenced real audio server. A signifi- 

Users, which claims priority to U.S. Provisional Patent cant drawback to such an arrangement is that the user must 

Application No. 60/034,128, filed Dec. 31, 1996, both of the lO download the real audio player program. This program must 

aforementioned applications arc herein incorporated by rcf- then be installed on the user's computer. This may cause a 

erence. number of problems for the user. For example, if the user is 

behind a firewall, or some security program, the client may 

BACKGROUND OF THE INVENTION ^ot be able to receive the broadcast from the server. 

1. Field of the Invention 15 Additionally, the installation of any program may have 
This invention relates to the field of network dehvcry of conflicts with other programs. The program has the disad- 

audio and video information. In particular, the invention vantage of being platform specific. This means that a dif- 

relates to a system for delivering audio and video informa- Cerent program must be developed and downloaded for each 

tion to multiple users ^yp^ computer that is to be used to access RealNetworks 

2. Background Information ^° broadcasts. Additionally, the broadcasters of the streaming 
^ , " ^, .._ , audio and video need to use the RealNetworks server, which 
•nie Internet enables many different ways of communi- ^ ^ ^^^^ broadcasters' World Wide Web server 

catmg.OrigmaUy. the Interact was used for the exchange of (^^0 referred to as the web server). Tliis increases the 

flies and electromc maU. As the capab.hues of the Internet broadcasters' security problems because now the broadcast- 

expand, other types of commumcations are enabled. ^ ^e concerned with two separate servers. 

Audio and video transinissions are an important area of ^^^^^ ^^ ^ ^y^^ ^^^^ ^^^^ 

the communicauons that the Internet enables. For example. communications is the MBone. Tht MBone is 

many technologies supportthe transmission of digital video ^ specialized communication network that allows for the 

and/or audio signals over the MeraeL An example of such ^isiribution of streaming video and audio signals to multiple 

a technology is Quicklime™ available from Apple 30 ^ specialized network is set up specifically to transmit 

Computer, Inc., of Cupertino Calrf. Quicktmie movies are ^^^^ communications. A significant drawback of this 

files that can be transmitted across the InterneL QuideUme ^ connected to the specialized 

provides both audio and video displays Many other fi e ^^^^^^ Additionally, useis will be required to have spe- 

formats aUow audio and video to be displayed on people s ^.^^^ ^^^^^ '^^^^ computers to listen to and watch 

computers. 35 ^3^^,^ transmissions. 

This paragraph describes an example use of a the Quick- . ■ .j . •• • . j 

time technology. Auser will have a browser application that , Aslreaming video system, not requiring a user to down- 

. . . . r' . -m. . S I * load a separate program, was developed for a .single user by 

resides on his/her computer. The computer, acting as a client r >. ,.r -ru - t,,««^™i 

under the direction of the browser application, wUl connect °hn Graharji of California. T^ts smgle u«sr broadest 

to vadous World Wide Web (web) sei^vers. Each web server 40 '^.^hnology allowed a web serve to serve a smgle streaming 

.« . . „ . . * 1 1 /tJT-KMi \ video sicnalto a sinele client. Although the user did not need 

will typically serve hypertext markup language (HTML) yi^^^^^^y^ b -J i 

ci / «u r * Tu fii Lu.A^ \..^u'r.i to download a plug-in to see the video, only one user was 

files to the chents. The files may include text, graphics „ , .u j . . . t« .t,v 

references, and references to speciiOized flies. Some of these ° ""^f.^ ^^e video stream at a tmie^ In this system 

specialized files can include audio and video information in informatuDn was captured from » c»inera and 

Quicktime form. THe clients can then play these audio and 45 f'^T"^' w.wc^"'' '^'^^ T ""P*"; 

. 1 J J - r^, • 1^- la ted in a MIME encoded mulupart data stream. The chcnt 

video files once they are downloaded using a QuickUmc " J" ; " j . ^ ^ e e ,u 

, . . , /. • w u iM- u u received this data stream and reconstructed frames of the 

plug-m, a helper application, or Quicktune capabilities built j • i • j 

into the browser application. A plug-in and a helper appli- igi a vi co. 

cation are described in greater detail below. Therefore, what is desired is a platform independent video 

Streaming audio and video, as a subset of all the types of 50 streaming system that does not reqmre the user to 

audio and video that can be transmitted over the Internet, download additional programs beyond the functionalities 

allow people to broadcast long and/or Uve video and audio ^ browser. 

transmissions across the Internet Streaming video and audio ^ ^ SUMMARY OF THE INVENTION 
is video and audio digital data that is transmitted on a 

continuous basis, A client can access the data stream and 55 A system and method of providing streaming audio and 

regenerate the video images and audio signal as they are video data to multiple users is described. In one 

being transmitted. Streaming technology is particularly embodiment, the system comprises a first client, a second 

helpfiil where the events are live, or where the files would client and a server. The first and second clients are executing 

be so large as to be a burden on the end users. Examples of browsers. The server can communicate with the two clients, 

where streaming technology is particularly useful are for the 60 The server concurrently provides streaming audio and video 

display of conferences, sporting events, radio broadcasts, data to both of the cUents. Importantly, the server does not 

television broadcasts, and the like. require the two browsers to use a plug-in or a helper 

RealNetworks, Inc. of Seattle, Wash., provides a system application to receive and use the streaming audio and video 

for transmitting streaming audio and video signals to users data. 

over the Internet. RealNetworks supplies a server that allows 65 In some embodiments of the invention, a browser causes 

multiple users to simultaneously receive streaming audio a client to request an HTML file from a web server. The 

and video. client receives the HTML file. The HTML file includes an 



05/28/2004, EAST Version: 1.4.1 



us 6,732,183 Bl 

3 4 

HTML tag that directs the browser to load one or more Clieai: a computer, process, program, or combination of 

applets from the server. The browser executes the applets computers, processes or programs, that can communicate 

caxising the browser to request streaming audio and video with a server. 

from the web server. That request may or may not include g^^^^ 3 computer, process, program, or combination of 

parameters givmg mformation about the type of request s computers, processes or programs, that can communicate 

being made. The web server associates a server process with ^ ^^^^^ ^^^^ j^e cUent can be executing on 

the request, given the parameters m the request. The web compulcr(s), 

server notifies real-time audio and video process that stream- „, . - . *t * * 1 * j 

J- J -J • £_ • J J T . *L Web server: a server for serving at least Internet related 

mg audio and video mformation is needed. In response to the 1 i_ rrm^ 

^. , ^. J. J -J . requests. Example web servers can serve HTML pages m 

notification, the real-time audio and video process stores in ^ ^ jjrJL. , ^ i- . o u 

jjj. J -J J*- 1 response to HTTP requests from cheats. Some web servers 

encoded audio and video data m a shared memory locauon. »v h j . i • j r * uT-mc 

*u L J J ■ can serve many different kinds of requests, e.g., H I IPS, 

The server process accesses the shared memory and mserts ^n^or FTP n ' o ' 
the audio and video data into one or more data streams. The 

cUent receives the data streams and reconstructs the audio Browser: an application, program, process, or combma- 

and video signals using only the capabilities of the browser. 15 ^f applications, programs or processes, that allow a 

In some embodiments, a separate stream and server process cUent to make a request of a web server and process the 

is used for each of the audio and video data. These embodi- results of the request. The browser may be part of a stand 

mcnts allow multiple clients to simultaneously receive the alone application or a set of programs that are integrated mto 

same audio and video data. operating system of the client. 

Other embodiments of the invention include a web server 20 Plug-in: plug-ins are external software programs that 

that can serve streaming audio and video information as well extend the capabihties of the browser in a specific way. For 

as perform more usual web server functions (such as, example, a plug-in can be used to give the browser the 

serving web pages, performing file transfers, supporting abiUty to play audio samples or view video movies from 

secure communications). These embodiments have the within the browser. 

advantage of allowing the broadcasters and the users to set 25 Helper application: like plug-ins, helper applications are 

up their security configurations for one web server, rather external software programs. The browser redirects some 

than two servers (a web server and a streaming audio and types of file to the helper applications. The helper appfica- 

video server). tions allow clients to process many different types of files on 

Although many details have been included in the descrip- the Internet. When the browser encounters a sound, image, 

tion and the figures, the invention is defined by the scope of 30 or video file, the browser hands off the data to the helper 

the claims. Only Umita tions found in those claims apply to applications to run or display the file, 

the invention. JavaScript: a stand-alone programming language built 

2 A BRIEF DESCRIPTION OF THE DRAWINGS many browsers. Primarily an extension to the Internet 

standard HTML language. 

The figures illusuate the mvcntion by way of example, 35 . . , . ^ r^^^u^A 

andnolto^uUon.Likereferencesmdicatesimilarelements. ^^7' » Prog^^ni-nuig language who^ programs (^ d 

ij- uj- . e applets) can travel over the Internet for use by chents. 

no. 1 Illustrates a system mcludmg one embodiment of appUcations are taUored for a 

a strcammg audio and video system for multiple users where .^^^^ j^^^ ^^^^^^ ^^^^^ 

chent computers do not need plug-ins or helper programs. {j^^ internet regardless of the type of client. Many browsers 

no. 2 iUustrates an example method of streammg audio -lO ^^j^^^ j^^^ capabUilies so Java applets require no instal- 

and video for multiple users. j^jj^j^ g^^^^pj ^^^^^ ^^^^^^ j^^^ JavaScript are inter- 

FIG. 3 illustrates a web page that a user can use to access changeable, 
a streaming audio and video broadcast. 

FIG. 4 illustrates a web page for selecting the bandwidth (b) System 

of the user's Internet conneaion. ^ .„ . • , j- u j- * r 

T-T^ e * * u u • * -J ,1 FIG- 1 illustrates a system mcluding one embodiment of 

FIG. 5 illustrates a web page having sureammg audio and • u j a a J^ .. ^ r-j 

^ r & & & jjjg mvention where audio and video is supphed over the 

J r Internet to multiple users. The following paragraphs first list 

FIG. 6 is a block diagram under another embodiment of ^^^^^^^ ^ ^^^^^^^^ ^^^^ interconnections, and 

the invention, iUustratmg a proxy switchmg system ^^^^-^^ ^^-^^ elements in greater detail. 

no. 7 is a flow chart for a process to use proxy switching paragraph lists the elements of FIG. 1. HG. 1 

for providmg streaming media to client terminals, under an i^^j^^es a system having three parts: a cUent side 100, a 

embodiment of the invcnUon. communications interface 180, and a server side 130. The 

3. THE DESCRlPnON client side 100 includes a client 112 and a chent 111. The 

The following sections describe embodiments of the 55 cUent 112 includes a browser 102 having a video display 

invention. The first section provides definitions that will help area 103. The chent 111 mcludes a browser 108 and a video 

in the understanding of the remaining sections. The second display area 104. The commumcations interface 180 

section shows an example system that supports various includes the Internet 185. The server side 130 mcludes a web 

embodiments of the invention. The third section describes an server 131, a shared memory 135, and a real-time server 

example method of using the invenUon. The fourth section ^0 140. The web seivcr 131 mcludes two processes, a process 

illusuates an actual use of the streaming audio and video ^38 and a process 139. The real-Ume server 140 mcludes a 

used in some embodiments of the invention. The last section video module 144, an audio module 146, a video proxy 148, 

reviews additional aUernaUve embodiments of the invention. and an audio proxy 149. The real-tune server 140 mterfaces 

with a number of other elements. These elements include a 

(a) Definitions ^ideo card 159, an input audio interface 162, and an HTTP 

The following definitions will help in the understanding connection to remote server 170. Various elements of FIG. 

of the following description. 1 have now been listed. 
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The following paragraphs describe ihe inierccnQeciions Similarly, the input audio interface 162 allows for ihe 

between the elements of FIG. l.Beginmng on the server side input of analog audio signals and converts this input to 

130, the video card 159 receives a video input 158 outputs digital audio signals. What is important is that the audio 

a digital video signal to the video module 144. Similarly, an module 146 receives a digitized audio signal that can be used 

input audio interface 162 receives a mike input 164 and/or 5 by the real-time server 140. The audio modxilc 146 may 

a Une input 166 and outputs a digital audio signal to the convert the digitized audio signal into any of a number of 

audio module 146. The HTTP connection to remote server formats, corresponding to any of a number of audio trans- 

170 receives a streaming audio and video data 175. The rates (examples include G.711 and G.723 audio 

HTTP connection to remote server 170 outputs the video compression formats). 

data to the video proxy module 148 and the audio data to the lo "TTP connection to remote server 170 represents an 

audio proxy module 149. The real-Ume server 140 uses the important advantage of one embodiment of the mvenUon. In 

data received by the various modules and stores portions of embodiment, the HTTP connecUon to remote server 170 

that data in the shared memory 135, after some manipulation ^l^o^s the server side 130 to forward broadcasts of audio 

of the data. The shared memory 135 is accessed by the web ^^"^ ^^^^o signals from other streammg audio and video 

server 131. The process 138 and the process 139 transmit 15 servers. In these uses, the server side 130 acts as a client to 

and receive data to the communications interface 180. Thus, ^^^ther server. The HTTP connection to remote server 170 

the couplings of the server side 130 have been described. In ^^^i^^ ^^^^ and audio signals bemg broadcast through 

some embodiments, the process 138 and the process 139 ^« ^^^^^n^* from another server. The HTTP connecUon 

correspond to HTTPD processes. ^^^^^^ server 170 provides the digital video mformaUon 

„ . . . ^ *oA 11 1* . J on from the other server to the video proxy module 148. 

The commumcauons in efface 180 allows the cheat side ^ ^.^ connection to remote server 170 pro- 

100 to commtimcate with the server side 130. The commti- ^.^^ ^^^^ j^,^ ^^^^ ^^^^ 

nications interface, and the Internet 185 m particular, sup- ^.^^ ^^^^ 

port many different types of connections by the chent Side . / . j .u. 

-J iin I 1 • ihcn supply the respective video and audio data to the 

100 and the server side 130. In the example communications ^^^j time server 140 

interface 180, the communications interface 180 includes ^ ' , . ^^/v . i- 

the Internet 185. In parUcular. in this example, the process TTe rea -time sender 140 represents an apphcauon, or se 

138 is communicating streaming audio and video data 176 °^ apphcaUons, execuUng on one or more computers that 

to the Internet 185. SimUarly, the process 139 is communi- P^-^P^f """'^ ^"^^ ^''^^ '^'''',1°' broadcasung to multiple 

eating the streaming audio and video data 178 to the Internet ^t'' The real-Ume «rver 140 

e» o (3j^^ ^jjg (J f ^jje vanous modules, processes the data, 

and stores the processed data in the shared memory 135. The 

On the client side 100, the cUent 112 is commumcatmg real-time server 140 can perform compression, and other 

the streaming audio and video data 176 with the Internet manipulations of the data, to reduce the processing burden 

185. SimUarly, the cUent 111 is commumcaling with the ^^^^j. example, in some embodiments 

Internet 185 to receive and manage the streammg audio and invention, the real-time server 140 receives digitized 

video data 178. The clients then communicate the infonna- ^-^^^ compresses that data into JPEG images, 

tion to their respective browser appUcations. The browser jpg^ -^^^^^ sequenced digital frames of video, 

applications the generate video images in their respective similarly, for the audio data, the real-time server 140 breaks 

video display areas. 3^^^^ information into one-half second time periods of 

Thus, the connections between the various elements of audio data (other embodiments use other time periods). 

FIG. 1 have been described. Now the various elements will These one-half second time periods of data are stored in the 

be described in greater detail in the following paragraphs. shared memory 135. The real-time server 140 can also 

An example method of using these elements is described compress the audio information into one of a number of 

below in relation to FIG. 2. various compressed audio signals (e.g., G.711 and/or G.723 

The server side 130 will be described first. 45 audio compression formats). In some embodiments of the 

The video input 158 represents a video signal that a user invention, the real-time server can broadcast audio and video 

of such a system wishes to broadcast to the clients on the from mulUple sources to mulUple clients, 

client side 100. The video input 158 can include analog The shared memory 135 represents a shared storage area 

signals representing video information. The video card 159 for use by the real-time server to store audio and video data 

digitizes the video input 158 to produce a digital video 50 for access by the web server 131. In one embodiment the 

image. Various embodiments of the invention include Sun shared memory 135 has a locking and semaphore usage 

video cards available from Sun Microsystems, of Mountain scheme to ensure that the real-time server 140 is not writing 

View, Calif, and Parallax XVldeo Xtra^" video cards, data into the shared memory 135 while the web server 131 

availablefromParallaxGraphicSjInc, of Santa Clara, Calif. is accessing that data. In some embodiments, the sema- 

However, what is important is that the real-time server 140, ss phores act as notifiers to indicate that new data in the shared 

and in particular the video module 144, receives some sort memory 135 is available for use by the web server 131. In 

of digitized video signal. The video module 144 is respon- some embodiments, the video data and the audio data are 

siblc for providing the real-time server 140 with the video stored in different shared memory locations, 

information that will be broadcast to the client side 100. The The web server 131 communicates data over the Internet 

video module 144 can convert the digitized video signals to 60 185 using one or more communications protocols. In some 

be of better use to the rest of the system. For example, the embodiments of the invention, these protocols include 

video module 144 may, if not done by the video card 159, HTTP (Hypertext Transfer Protocol), TCP (Transmission 

convert digital video data into a sequence of JPEG digital Control Protocol) and UDP (User Datagram Protocol). The 

video frames. In any case, what is important is that the web server 131 represents an application, including one or 

real-time server 140 receives digital video information in a 65 more processes, for communicating over the Internet 185. In 

format that it can use (example formats include, JPEG, one embodiment, the web server 131 includes an Apache 

MPEG, GIF, and AVT). web server. Each of the processes in the web server 131 
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represents one or more processes for serving streaming 
audio and video data to the client side 100. In some 
embodiments, the web server 131 transmits the video data as 
a multipart MEME (multi-purpose Internet mail extensions) 
encoded file for decoding directly by the browser or as 5 
compressed video information for decoding by an applet run 
in the browser. The web server 131 transmits the audio data 
as compressed audio data for decoding by an applet run in 
the browser. 

In some embodiments, the web server 131 initiates a 
separate process for each audio and video connection made 
from client side 100. Thus, for one client receiving stream- 
ing audio and video data, two processes would be started 
within the web server 131. The first process would supply 
video data and the second process would supply audio data. 
The processes access the shared memory 135 and serve the 
data across the Internet to the respective client. 

In some embodiments, the web server 131 initiates at least 
one process for each client This provides important advan- 
tages in some embodiments of the invention. In particular, ^ 
because the web server 131 is serving the data directly 
through processes it created, server side 130 users need not 
worry about security issues beyond those already faced with 
their web server 131. Thus, these embodiments of the 
invention have a lower chance of interfering with client side ^ 
100 fire walls and have a lower chance of having a server 
side 130 security problem. 

Other embodiments of the invention include separate 
Common Gateway Interface (CGI) programs for audio and 
video. These CGI programs are used by the web server 131 3Q 
to serve the streaming audio and video data. These CGI 
programs are not necessarily integrated as tightly to the web 
server 131 as the process 138 and the process 139. However, 
a CGI program allows for the easy extension of many 
different types of web servers. 35 

The communications interface 180 permits communica- 
tions between the server side 130 and the client side 100. In 
this example, the Internet 185 supports the communications. 
Other embodiments of the invention support other commu- 
nications interfaces. For example, the Internet 185 can be 40 
replaced by a local area network, a wide are network, a 
proprietary telecommunications and networking 
infrastructure, or some other communications interface. 
What is important is that the server side 130 can commu- 
nicate with the client side 100. The communications inter- 45 
face 180 can also include combinations of the above 
described technologies. For example, the Internet 185 can 
include a web server to which the clients on the client side 
communicate through to access the Internet 185. The cHent 
side 100 can be on a local area network that is connected 50 
through a server, or router, to the Internet 185. 

The client side 100 represents the consumers of the 
streaming audio and video data. In this example, the two 
clients are receiving separated streaming audio and video 
data signals. Other embodiments of the invention support 55 
many more clients. 

The client 112 represents a computer, such as a PC 
compatible computer, running a browser application 102. 
For video display, the browser application 102 can include 
a Netscape Navigator™ or Commimicator™ program for 60 
"multipart/xmixed-replace MIME type video," or a 
Microsoft Internet Explorer"*"" 3.0 or later for a Java based 
video transmission. In some embodiments, the Java based 
video transmission applet parses the multipart/x-mixed- 
replace MIME type video. For audio, the browser applica- 65 
tion 102 can include any browser that supports Java and/or 
JavaScript. 



The browser application 102 is responsible for receiving 
the streaming audio and video data 176 and reconstructing 
an audio and video signal suitable for the end user. In this 
example, the video display area 103 displays the rccon- 
sUiicted video information received from the video input 
158 at the real-time server 140. Similarly, the client 111 is 
executing the browser 108. The browser 108 is displaying 
the same video signal in the video display area 104. The 
client 111 represents another computer executing a browser 
application. 

Various embodiments of the invention have modifications 
to the system shown in FIG. 1. Some of these variations are 
described in this paragraph. For example, the client lU and 
the client 112 can be the same computer or be different 
computers. The ctients can be on the same local area 
network or be on completely different local area networks. 
There can be many more clients receiving the information 
from the client side 100. Additionally, as shown with the 
HTTP connection to remote server 170, a real-time server 
140 can appear on the client side 100 to distribute data to 
other clients. 

Note that portions of the system, and embodiments of the 
invention, are sets of computer programs or computer that 
can be stored on computer usable media such as floppy 
disks, hard drives, CD ROMs, Zip disks, etc. 

Thus, an example system supporting streaming audio and 
video data for multiple users has been described. 

(c) Example Method 

FIG. 2 illustrates one example of a method of broadcast- 
ing streaming audio and video data to multiple users. This 
example method could be executed on the system of FIG. 1. 
In this example, the client 112 will initiate a request to 
receive streaming audio and video data 176 from the server 
side 130. The client 112 will display the video data in the 
video display area 103 and will play the audio data for the 
user. Importantiy, the client 112 can play audio and video 
without the need of a plug-in or helper application. As will 
be shown, the audio and video play as part of a transparent 
process of the client 112 loading a web page from the server 
131. 

At block 210, the cUent 112 initiates an HTTP request 
from the web server 131. This could be the result of the 
browser 102 receiving and displaying an HIML (hypertext 
markup language) page including a link that will initiate 
streaming audio and video. The user would have selected 
this link which will then result in the browser 102 making 
the connection to the web server 131. 

As a result of the connection, at block 220, the web server 
131 suppUes a Java applet for decoding audio data and that 
will help in the display of video information. These instruc- 
tions could be supplied as two separate Java applets or one 
combined Java applet. 

At block 230, the client executes the Java applets. The 
video display portion of the applet initiates a request of the 
web server 131 for a HTTP transmission of a multipart/ 
mixed replace MIME encoded video (or in other 
embodiments, video data for Java decoding). Also at block 
230, the audio Java applet makes a similar request of the web 
server 131. The requests can include optional information 
such as desired video frame rates or audio rates. The requests 
include the URI (universal resource indicator) indicating the 
particular audio or video streaming information to be served. 

The following will describe the audio serving by the web 
server 131 and eventual decoding by the client 112. The 
video serving will be described after the audio. 
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At block 240, in response lo executing the Java applet, the 
client makes an audio request of the web server 131. This is 
done through the Java applet which supplies a universal 
resource indicator to the web server 131. The universal 
resource indicator can indicate the audio stream thai is being 5 
requested. This request looks, to the client 112, like a file 
download request. The web server 131 responds accordingly 
to this request by beginning to transfer the streaming audio 
information. Importantly, the client 112 does not need to 
know that the file is a streaming audio or video signal that 10 
is essentially never ending. 

In response to the client request, the web server 131, and 
in particular the process 138, makes an audio data request of 
the shared memory 135. Note, if the process 138 had not 
been created, the web server 131 creates the process (in ^5 
some embodiments, the web server 131 creates a separate 
process for each of the audio and video data streams). The 
audio data request of the shared memory is done by the web 
server 131, and in particular by the process 138, by notifying 
the real-time server 140 that audio information is requested. 20 
In subsequent iterations, the web server 131 need not make 
the explicit request for the audio information. Once 
requested, the real-time server 140 will continue to provide 
audio information, in these embodiments, until it is told to 
stop. ^ 

Id any case, the real-time server 140, in response to the 
request, prepares audio information and writes this infor- 
mation into the shared memory 135. In various embodi- 
ments of the invention, the real-time server 140 prepares the 
audio data by breaking the audio information into time 
periods. This audio information is also compressed into 
various sets of compressed data corresponding to different 
audio rates. Higher audio rates corre^ond to better quality 
audio signals. In these embodiments of the invention, the 
real-time server 140 writes the data for the various audio 
rates to the diared memory 135, thereby reducing the work 
load of the web server 131. Different web server 131 
processes will require different audio rates depending on 
their connections to their respective clients. By storing the 
information corresponding to the different audio rates into 
the shared memory 135, each process can access the desired 
audio rate data from the shared memory 135. Thus, the web 
server 131 need not calculate the compressed audio data for 
each process within the web server 131. 

The process 138, through the web server 131, now 
transmits the data accessed from the shared memory 135. 
This corresponds to block 246. 

At block 248, the client 112 receives the compressed 
audio data. The client decompresses the audio data as 
commanded by Java audio applet, and plays the audio 
information through the audio system of the chent 112. 

At block 241, a test is made to determine whether the web 
server 131 should continue broadcasting the streaming audio 
information to the client 112. This test is made by deter- 55 
mining whether the client 112 has broken the connection to 
the web server 131. 

The web server will continue serving the data as long as 
the client 112 is connected to the web server 131 through the 
Java audio applet. 50 

Note, importantly, the user at the client 112 has not had to 
download any additional plug-ins or helper programs lo play 
the streaming audio information. 

'Riming to the video broadcasting, at block 252, the web 
server 131 makes a video information request to the real- 65 
time server 140. The real-time server 140 lakes each video 
frame from the video module 144, or the video proxy 



45 



module 148, and compresses that information into a JPEG 
image. In some embodiments of the invention, the video 
card 159 provides the images as JPEGs. The requesting 
procedure is similar to that followed in the audio request 
block 240. 

The real-time server 140, at block 254, writes one JPEG 
frame into the shared memory 135. The process 138 
accesses the shared memory 135 lo retrieve the JPEG frame 
and transmit thai frame to the client 112. At block 256, the 
process also formats the JPEG as part of a multipart MIME 
encoded file. 

At block 258, the client 112, using the capabilities of the 
browser 102, decompresses the video data from the MIME 
encoded format, and the JPEG encoded form, and creates the 
video display 103. 

At block 251, the web server 131 determines whether the 
video Java applet is still requesting video frames. Block 254 
through block 251 are then repeated. The result of these 
blocks is that multiple frames of video information is 
displayed in the video display 103. Thus, the user has the 
perception of a video display at the client 112. 

Note, by the time the audio information is played from the 
previous audio transmission, a new audio transmission has 
been received and decompressed. Thus, the client 112 will 
have a continuous audio signal being presented to the user. 

If it is the case that the audio, or the video, information is 
not being received by the client 112 at a sufficient data rate, 
the corresponding Java applet, in some embodiments of the 
invention, can request a different rate of transmission. The 
Java applet can request a lower rate corresponding to a lower 
audio or video signal, that will more appropriately match the 
bandwidth availability of the client 112. 

One advantage of the system of FIG. 1 is that if the web 
server 131 becomes heavily loaded, the video frame rate is 
automatically reduced. This is done by ensuring that the 
audio processes take priority over the video processes. If a 
video process cannot access the shared memory 135 in 
sufficient lime, that video frame will simply not be trans- 
mitted to the client 112. However, the corresponding audio 
process should have an opportunity to transmit the audio 
information. 

As has been seen by the above discussion, the user has not 
been required to download any plug-ins or use any helper 
applications in the receiving of the streaming audio and 
video data. Additionally, the web server 131 is able to 
execute this example method for multiple clients. Each 
client would have a corresponding set of processes in the 
web server 131. The number of processes is only limited to 
the number of connections that can be supported by the web 
server 131. As has been noted, some of the work that would 
normally be performed by the web server 131 has been 
moved into the real-iime server 140 to reduce the load on the 
web server 131. 

(d) Example Video Display 

FIG. 3 through FIG. 5 represent the interface presented to 
a user using a browser 102 on a client 112. In this example, 
the user is using a standard PC with a standard Netscape 
Communicator'*^" 4.0 browser application. FIG. 3 includes a 
browser window 302 with a cursor positioned over a trans- 
mission selection link 304. The transmission selection link 
304 corresponds lo a request for transmission of streaming 
audio and video data. FIG. 4 illustrates the result of the 
selection of the transmission selection link 304. The user is 
presented wiih a number of coimectioD speeds to select 
from. The user selects the speed associated with his or her 
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connection speed to the Internet 185, In this example, the The proxy switching source 635 includes a web server 
user is selecting a Tl or better connection speed. In one 632 to communicate with client terminals and other servers 
embodiment of the invention, the Java applets are control- over the Internet. The proxy switching source 635 may also 
ling the presentation of the various selection speeds. This is function as a streaming media source for client terminals 
shown as a bandwidth selection link 404. 5 610. To this end, proxy switching source 635 may include an 
FIG. 5 illustrates the result of the "Tl or higher'* band- audio module 636 to receive and digitize audio input for 
width selection link being chosen. A digital video display broadcasting streaming audio media to client terminals 610. 
502 is shown in the browser window 302. The chent The proxy switching source 635 may also include a video 
computer executing, the Netscape Communicator browser, module 638 to receive and digitize video input for broad- 
is playing an audio signal corresponding to the digital video lO casting streaming video media to the client terminals 610. 
display 502. In this example, a user is allowed to modify the The proxy switching source 635 also inchides proxy switch- 
video rate, shown in frames per second, by selecting one of ing module 634. The proxy switching module 634 commu- 
the video rate selectors 510. A result of selecting one of the nicates with other servers to receive streaming audio and/or 
various video rate selectors 510 would be that the corre- video media. In particular, the proxy switching module 634 
spending video Java applet will communicate with its cor- 15 communicates with web servers 622 of the streaming media 
responding process in the web server 131. The Java applet sources 620A and 620B to receive streaming media via the 
would request a change in the video frame per second rate Internet 605. For proxy switching source 635, shared 
being supplied by the process in the web server 131. memory 631 stores streaming media received from video 
Similarly, a set of audio rate selectors 512 allow the user to module 638, audio module 636, or through media source 
select a higher or lower quality audio signal. The corre- 620A and 620B via proxy switching module 634. The web 
sponding Java audio applet would communicate with its server 632 of proxy switching source 635 transmits stream- 
corresponding process in the web server 131 to request the ing media that is first stored in shared memory 631. 
change in the audio rate. The streaming media sources 620A and 620B each 

/ \ Ajj-.- 117 u j_ * include web server 622. Each streaming media source 620A, 

(e) AddiUonal Embodiments . j » ■ i j j i rio a 

^ ^ 25 520B is assumed to include a video module 628, an audio 

Thus a system for broadcasting streaming video and audio module 626, and a proxy switching module 624. As with 

information to multiple users has been described. However, p^xy switching source 635, the audio module 626 receives 

various embodiments of the invention include modifications digitizes audio input. The video module 628 receives 

and optional additions to the system. and digitizes video input. 

It is possible for the real-time server 140 to be receiving 30 ^^^^^ switching module 624 enables sources 620A, 

and U-ansmitting multiple different video and audio signals g20B to each receive streaming audio and video media from 

through the web server 131. Additionally, multiple web ^ther sources (not shown). A shared memory 621 stores 

servers could be accessing the informaUon from the real- streaming media data for web server 622. In this way, an 

dme server 140. Additionally, multiple real-time servers 140 embodiment may provide for each source 620A, 620B to 

could be providing information for the web server 131. 35 alternatively function as proxy switching source 635. 

In some embodiments of the invention, users can select embodiment, administrator 625 is a client terminal 

cither the audio stream, the video stream or both the audio ^^^^^^ ^^8. The administrator 625 may also include 

and video stream from a server. Similarly, some users can ^ ^-^^ streaming media from the selected 

select one of these sets of strcanjs while other users se ect ^^^^/^ /20. In an embodiment, administrator 625 includes 

other sets of streams Additionally, some users can select ^0 ^^^^^ .^^^^^^^^ ^ ^^^^^ streaming media 

different rates for audio or video information than other ^^^^^^^ g^OA, 620B for broadcast to cUent terminals 610. 

users. In such systems, the real-time server 140 supplies the ^^^^ j ^-^^^ -^^^^^^^ j streaming 

data correspondmg to the requested rales to the shared ^^^^ ^.^^^ ^^^.^ ^^^^ ^^^^ ^^^^^ 620A and 620B. 

memory 135. For example, if one user ls requestmg a firet ^.^^^ .^^^^^^^ ^^j^^^ ^ ^^^^^^^ 

audio rate and a second i^er is requestmg a second audio 45 ^^^^ ^ poim-and-click mechanism or selection icon, to 

rate, the shared memory 135 would store audio data for both ^^^^^^ ^^.^^^ administrator 625 to select one of the 

audio rates of compression. In some embodiments, the web streaming media sources 620A, 620B for broadcast to cUent 

server 131 notifies the real-time server 140 when a particular terminals 610 

audio rate is not required by any of the processes anymore. ' . ^ r 

50 Initially, the administrator 625 selects, for example, media 

(f) Proxy Switching source 620A. The selection is passed to proxy switching 

FIG. 6 illustrates a system 600 under another embodiment source 635. The proxy switching soiu-ce 635 establishes 

of die invention. The system 600 enables viewers on chent communications with the selected media source 620A. 

terminals 610 to experience streaming media broadcast from When media source 620A is selected by administrator 625, 

multiple sources 620A on a network such as the Internet, ss the proxy switching source 635 couples to a channel 617 for 

With system 600, client terminals 610 may be switched receiving streaming media from media source 620A. The 

seamlessly between media sources. Specifically, users of shared memory 631 stores the streaming media from source 

chent terminals 610 can be switched between sources of 620A. The streaming media stored in shared memory 631 is 

streaming media without experiencing an interruplion that subsequently signaled by web server 632 to client terminals 

would otherwise result from breaking a connection with one 60 i^^*^ administrator 625). The web server 632 signals the 

streaming media source 620 A to receive streaming media streaming media from the media source 620A to the client 

from another source 620B. tenninals 610 on a communication channel 627 available 

In an embodiment, system 600 includes adminisU-ator 625 through the Internet 605. 

coupled to proxy switching source 635 across the Internet The administrator 625 may subsequendy select source 

605. The proxy switching source 635 communicates over the 65 620B. A switching control signal is transmitted to web server 

Internet 605 with other streaming media sources 62 OA and 632 via Internet 605 to signal the switch. The switch control 

620B. causes proxy switching module switches to channel 619 to 
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receive streaming media from source 620B. The shared vided from either administrator 625, or from one of the 

memory 631 tbeo receives streaming media from source client terminals 610. The command is received by the server 

620B. The web server module 632 accesses the streaming 632 of the proxy switching source 635. 

media from shared memory 631. The web server 632 then server 632 of the proxy switching source 635 

signals the streaming media from source 620B to client j communicates the command to switch media sources to the 

termmals MO on the commumcauon channel 627, once ^^^^ ^ 

streamuig media nom source 620A that remaining ID shared r i . /nnrx l 

memory 631 is transmitted to client terminals 610. TTie web ^nifoma resource locator (URL) which instructs the web 

server 632 does not disconnect or break the connection of ^^^^^ ^« ^^^^^^^^ PiP^^^f module 634 to switch to the 

communication channel 627 while proxy server module 634 media source 620B. Tlie URLsignalcd as the cornmand 

makestheswitchfromsource620Ato620B.niewebserver '° parameters to specify the switch. One format 

632 seainlessly transmits the streaming media from source ?. pai^meters to cause the switch to a new 

620B. -nie client terminals 610 maintain their connections "^^^'^ ^^^^''^^ ^ 

with web server 632, and the web server 632 is not inter- www.<hostingserver>.com/<file typo? server -<viewing 

rupted in signaling streaming media after switching to channe number>% source a<sourcelocation>: 

source 620B. Thus, once administrator 625 switches from ^ <portnumber>-<channel number> 

source 620A to source 620B, clients 610 are all seamlessly 1° ^he URL illustration, the new media source is provided 

switched together from source 620A to source 620B. The as a new channel to clients 610. 

communication channel 627 between client terminals 610 1° step 730, the proxy module switches to a different 

and web server 632 remains intact. The effect is that all of streaming media source. For example, the proxy module 

the client terminals 610 switch streaming media sources switches to receive streaming media from source 620B, 

620A and 620B nearly simultaneously, without an interrup- rather than 620A. 

tion or break in experiencing streaming media. step 740, one or more of the participants in the audience 

While system 600 iUustrates administrator 625 as signal- experience streaming media from the new source. If, for 

ing a switch to occur from source 620 A to source 620B, „ example, adramistrator 625 makes the request, then all of the 

other embodiments may provide that one or more of the ^"dience experiences streaming media from the new source. 

cUent terminals 610 can signal the switch to proxy switching AlternaUvely, one of the participants may make an indi- 

source 635. In these embodiments, web server module 632 ^idual command, in which case only that chent terminal 610 

of proxy switching source 635 receives the switch request ^ coupled to receive sU-eaming media from the new source 

from one of the clients 610. That client terminal 610 making ,n ^^OB. .... , ... 

the switch request may be treated as an administrator, in an alternative mentioned with FIG. 6, proxy switching 

which case all of the other clients are switched to the source 635 may be switched to one of the sources 620A or 

requested source together. Alternatively, the client terminal 620B. A command to cause a new host for streammg media 

610 making the switch request may be switched to a new delivery may be provided as a URL with parameters such as: 

source individually. As still another alternative, one or more 35 www.<hostingserver>.com/<file type>? server =<viewing 

of the client terminals 610 may make the request to switch channel nuraber>% source- www. <ncwhost>. com: 

media sources to administrator 625. The administrator 625 <portnumber>-<channelnumber> 
then chooses the response to the request. 

In an embodiment shown by FIG, 6, the streaming media (s) Examples of Proxy Switching 

sources 620A and 620B are provided to alternatively func- 40 As one example, system 600 may provide a virtual 

tion as proxy sources for other streaming media sources. seminar or symposium for an audience of users operating 

Thus, in the example shown, each source 620A, 620B may individual client terminals 610. The speakers of the seminar 

route streaming media from one or more other sources, ^ay be recorded in real-time through audio and video 

Further, each of the steaming media sources 620A, 62 OB digitizers located on terminals or servers, forming streaming 

may act as proxy switching sources, so as to switch between 45 media sources 620A and 620B. For this example, it is 

other sources (not shown) feeding media to the proxy assumed that system 600 is operated over the Internet. Since 

modules 624 of the respective sources 620A, 620B. The each speaker may be located in a different location, the 

administrator 625 may access the web servers 622 of the media stream resulting from a broadcast by each speaker 

sources 620Aand 620B to cause streaming media received may be directed to proxy switching source 635. When a 

from one source (not shown) by respective proxy modules 50 presentation by one speaker is completed, an administrator 

624 to be switch to another source (not shown). One of the ^25 can switch the audience to a new speaker. The switch 

advantages provided by this embodiment is that system 600 may be made seamlessly from the audience's point of view, 

is scalable, allowing both proxy streaming media delivery even though each client terminal comprising the audience is 

and switching on a large scale. located remote from the rest of the audience. The adminis- 

FIG. 7 illustrates a flow process for operating proxy 55 trator 625 may also use the seamless switching to enable a 

switching, using a system such as shown by FIG. 6. In step discussion to occur between speakers. For example, a 

710, an audience is provided streaming media from a first speaker can indicate to administrator 625 a desire to respond 

source. The audience may comprise client terminals 610. to a statement by another speaker. The administrator 625 

The streaming media may include audio media and/or video may then cause the switch to occur, so that the new speaker 

media. The streaming media may originate from proxy 60 can address the audience, before switching back to the old 

switching source 635, or from one of the other sources 620A speaker. The administrator 625 can make the switch without 

and 620B. breaking the connection (communication channel 627) with 

In step 720, a command is provided to proxy switching the audience. As a result, the audience does not experience 
source 635 to switch sources from, for example, source a broadcast interruption when a switch is made to a new 
620A to source 620B. The command may alternatively 65 speaker. In this manner, the virtual seminar can be made to 
request to switch from proxy switching source 635 to either simulate a real seminar, or a closed network analog broad- 
source 620A or source 620B. The command may be pro- cast of a seminar. However, system 600 enables the Internet 
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to be iised in a manner that enables the audience and 
speakers to be located remote from one another. 

As another example, a classroom may use system 600 as 
an educational tool. The client terminals may be operated by 
students. The administrator 625 may be a teacher, providing ^ 
the students with an educational streaming media content. 
The system 600 allows the teacher to use client terminals 
610 to show each student the same streaming media content. 
The teacher may then switch the whole classroom to a new 
streaming media source, rather than require the students to lo 
make the switch independently. 

(h) Conclusion 

The foregoing description of various embodiments of the 
invention has been presented for purposes of illustration and 
description. It is not intended to limit the invention to the 
precise forms disclosed. Many modifications and equivalent 
arrangements will be apparent. 

What is claimed is: 20 

1. A system for providing streaming media to a plurality 
of clients over a network, the system comprising: 

a plurality of streaming media sources coupled to the 
network; 

a network server module that couples to the plurality of 25 
clients to signal streaming media from at least a first 
streaming media source in the plurality of streaming 
media sources to each of the plurality of clients using 
a first channel; 

a switching terminal to signal a switch request to the 
network server module, the switch request requesting 
streaming media from a second streaming media source 
in the plurality of streaming media sources; and 

a proxy module that forwards streaming media from the 
first streaming media source to the network server 



module, the proxy module being communicatable with 
the network server module to be signaled the switch 
request, and in response to the switch request, the proxy 
module forwarding streaming media from the second 
streaming media source to the network server module 
to facilitate seamless transition on the first chaimel 
from the first streaming media source to the second 
streaming media source. 

2. The system of claim 1, wherein the proxy module 
forwards the streaming media from the second streaming 
media source to the network server module while the 
network server module signals streaming media from the 
first streaming media source to the plurality of client termi- 
nals. 

3. The system of claim 1, further comprising a memory 
that stores streaming media forwarded from the proxy 
module for the network server module, the network server 
module accessing the memory to receive the streaming 
media forwarded from the proxy module. 

4. The system of claim 3, wherein the proxy module 
forwards the streaming media from the second streaming 
media source to the network server module while the 
network server module is accessing the streaming media 
from the first streaming media source from the memory. 

5. The system of claim 1, wherein the plurality of stream- 
ing media sources coupled to the network each include a 
proxy module to receive streaming media from other stream- 
ing media sources to facilitate large scale proxy streaming 
media delivery. 

6. The system of claim 1, wherein the switching terminal 
is an administrator for the plurality of client terminals. 

7. The system of claim 1, wherein the switching terminal 
is one of the plurality of client terminals that receives 
streaming media from the network server module. 
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